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ABSTRACT 

We present the near-infrared luminosity and stellar mass functions of galaxies in the 
core of the Shapley supercluster at z=0.048, based on new K-band observations car- 
ried out at the United Kingdom Infra-Red Telescope with the Wide Field Infrared 
Camera in conjunction with B- and i?-band photometry from the Shapley Optical 
Survey, and including a subsample (^650 galaxies) of spectroscopically confirmed su- 
percluster members. These data sets allow us to investigate the supercluster galaxy 
population down to and 7W=1O 8 - 75 M . For the overall 3 deg 2 field the if-band 

luminosity function (LF) is described by a Schechter function with M^-=-24.96±0.10 
and a=-I.42±0.03, a significantly steeper faint-end slope than that observed in field 
regions. We investigate the effect of environment by deriving the LF in three regions 
selected according to the local galaxy density, and observe a significant (2a) increase 
in the faint-end slope going from the high- (a=-1.33) to the low-density (a=-1.49) 
environments, while a faint-end upturn at M#- >-21 becomes increasingly apparent 
in the lower density regions. The galaxy stellar mass function (SMF) is fitted well 
by a Schechter function with logi (7W*) = 11.16±0.04 and a=-1.20±0.02. The SMF 
of supercluster galaxies is also characterised by an excess of massive galaxies that 
are associated to the brightest cluster galaxies. While the value of M* depends on 
environment increasing by 0.2 dex from low- to high-density regions, the slope of the 
galaxy SMF does not vary with the environment. By comparing our findings with 
cosmological simulations, we conclude that the environmental dependences of the LF 
are not primary due to variations in the merging histories, but to processes which 
are not treated in the semi-analytical models, such as tidal stripping or harassment. 
In field regions the SMF shows a sharp upturn below 7W = I0 9 M Q , close to our mass 
limit, suggesting that the upturns seen in our K-h&ud LFs, but not in the SMF, are 
due to this dwarf population. The environmental variations seen in the faint-end of 
the if-band LF suggests that these dwarf galaxies, which are easier to strip than 
their more massive counterparts, are affected by tidal/gas stripping upon entering the 
supercluster environment. 

Key words: Galaxies: clusters: general — Galaxies: clusters: individual: Shapley 
supercluster — Galaxies: photometry — Galaxies: luminosity function, mass function 
— Galaxies: stellar content. 



1 INTRODUCTION 

The properties and evolution of galaxies are stro ngly related 
to th eir environmen t (e.g . Blanton et al. l2005al ; Rines et al. 
120051 : Baldry et al. 200t|), through the mass and merging 
histories of their host dark matter halos, and the i mpact of 
different physical mechanisms (e.g. Treu et al. l2003l ) that are 
linked in various ways to the local galaxy density and the 



properties of the intergalactic medium. In the local Universe 
this environmental dependence has been investigated and 
observed in the distribution of galaxy luminosities and stel- 
lar masses, providing constraints on the assembly of galaxies 
over cosmic time (see below). 

Since the NIR light is dominated by established old stel- 
lar populations rather than by recent star-formation activ- 
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ity, the NIR LF can be considered as reliable estim ator of 
the stellar mass function (SMF, Gavazzi et al. Il996l ; Bell & 
de Jong|200l|) and the shape of the NIR LF co nstrai ns the 
scenarios of galaxy formation (e.g. Benson et al. [2003), with 
different energetic feedback processes from supernovae and 
AGN required to simulta neously fit the LFs at the faint and 
bright ends respectively (|Bower et al.ll2006h . Early versions 
of the hierarchical galaxy formation model predicted a de- 
crease of the abundan ce of massive galaxies with redshift 
(Kauffmann & Chariot 1 19981 ) which have continued forming 
until recent times through processes of merging and accre- 
tion, and a steep mass function due to the presence of a 
large number of faint dwarf galaxies witnessing the small 
dark halo formation in the early universe (e.g. Kauffmann 
et al. ll993h . 

The NIR LFs observed at different redshifts turn out 
to be well described by Schechter functions, although there 
is some evidence for an excess of bright galaxies with re- 
spect to the best fitting Schechter function (e.g. Jones et al. 
2006), while a faint-end upturn has al so bee n observed in 
some cases (e.g. De Prop ris & Pritchet ll99Sl ; Balogh et al. 
l200ll : Jenkins et al. [2007) . The results supp orting the hier- 
archical scenario (e.g. Kauffmann & Chariot 1998) are con- 
trasted b y othe r works that bring it into que stion (Kodama 
& Bower 120031 : De Propris et al. 1 19991 : 120071 ) or canno t in- 
terpret univocally their findings (e.g. Drory et al. ]2003t ) ■ For 
instance, there is a consensus that the evolution of the char- 
acteristic absolute luminosity M* for both field and cluster 
galaxies can be described by a passively evolving population 
form ed in a single burst at redshift 2=1.5- 2 (e.g. Lin et al. 
120061 : Wake et al. 120061 : De P ropris et al. 11999 : l2007h . On 
the other hand, Drory et al. ()2003h attribute the observed 
evolution in the A"-band LF of cluster galaxies, either to a 
change in the mass-to-light ratio alone (i.e. passive evolu- 
tion), or to a combination of changes in M/L and stellar 
mass which could be due to star formation and/or to merg- 
ing or accretion. Recent semi-analytic models incorporating 
AGN feedback have been able to reprodu ce we ll the evolu- 
tion of the SMF over 0<z<5 (Bower et al. 120061 ). predicting 
the observed population of massive galaxies at z>2. How- 
ever, in order to properly use the NIR, LF to disentangle 
between possible scenarios of galaxy evolution, one has to 
consider the dependence of the LF on environment, galaxy 
colour, spectral type and morphology. 

The LF of field galaxies at NIR wavebands has been 
primarily investigated through the Two Micron All Sky Sur- 
ve jEI (2MASS) often complemented by either optical pho- 
tometry from the Sloan Digital Sky SurvejQ (SDSS) or wide- 
field spectroscopic surveys such as the 2dF Galaxy Red- 
shift Survej0 (2 dFGR S). These works (e.g. Kochanek et al. 
l200ll ; Cole et al. l200ll ) agree in describing the NIR LFs with 
Schechter functions characterised by a rather flat faint-end 
slope a from -0.77 to -0.96 and which are in dependent of 
the morphological type (Kochanek et al. l200ll ). in contrast 
with that found in optical surveys where the faint-end slope 
is s teepe r for late-type galaxies. On the contrary, Bell et 
al. (2003) found that NIR LF has a brighter characteristic 
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luminosity and shallower slope for early-type galaxi es wit h 
respect to the later types. More recently, Jones et al. (2006), 
using the 6dF Galaxy Redshift Surve^O (6dFGRS, Jones et 
al. 12004 ) derived NIR LFs for field galaxies 1-2 mag deeper 
in absolute magnitude with respect to the previous works. 
They found that the Schechter function is not ideal to repro- 
duce the data since it cannot match the bright- and faint-end 
simultaneously, due to an excess of galaxies at magnitudes 
brighter than M*. 

Balogh et al. ([200 ll ) investigated the dependence of 
the infrared galaxy luminosity function and the associated 
galaxy SMF on environment and spectral type by means of 
2MASS an d Las Campanas Redshift Survey (LCRS, Sheet- 
man et al. [1996J) for galaxies brighter that Mj=-19mag. 
In field environments the LF of galaxies with emission lines 
turns out to have a much steeper faint-end slope (a=-1.39) 
compared to that of galaxies without emission lines (a=- 
0.59). On the other hand, in the cluster environment, even 
the non-emission line galaxies have a steep faint-end LF 
(a=-1.22). This difference is almost entirely due to the non- 
emission line galaxies which dominate the cluster popula- 
tion, and present a slope close to that of the overall field. 
Thus, they suggested that the cluster population is built 
up by accreting field galaxies with little effect other than 
the cessation of star formation. Differences in the shape of 
the LF for late- and early-t ype cluster galaxies has been 
found by Huang et al. (12IIII3:: the late- type galaxies having 
a systematically fainter M* and steeper faint-end slope. A 
possible faint-end upturn in the JJ -band LF was already sug- 
gested by De Propris et al. (|l998l . see also Andreon & Pello 
l200d ) for the Coma cluster outlining the steep trend of dwarf 
galaxies down to M*+5, even if they did not provide a pre- 
cise estimate of the faint-end slope because of possible field 
contaminations. An increase of the faint-end slope of Coma 
was recently observed at 3.6 /jm by Jenkins et al. (|2007l ) in- 
dicating a large number of faint red galaxies. However, Rines 
& Geller ((2003) found no such upturn for Virgo, based on a 
fully spectroscopically confirmed sample, and suggested that 
many of the photometrically-selected red sequence galaxies 
which contribute to the upturns seen in other clusters are 
background galaxies. 

Finally, the tight correlation between the total galaxy 
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NIR luminosity and t he clu ster binding mass (Lin et al. 
120041 : Ramella et al. |2004 ) allows to probe that the global 
cluster A-band mass-t o-light ratio decreases with cluster ra- 
dius (Rines et al. l2004 ) showing that the environment affects 
the shape of the LF also within the clusters. 

We note that most of the previous works are based on 
the 2MASS data which has a detection sensitivity (10a) 
of A"=13.1mag for extended source s (Co le et al. 1200 ll . 
A=13.57mag according to Bell et al. [2003), limiting stud- 
ies of the environmental impact on the NIR LF to only a 
sample of local clusters (e.g. Vi rgo), or limiting to magni- 
tudes <M*+2 (e.g. Rines et al. |2004| ). However, the dom- 
inant processes that quench star-formation, and therefore 
transform galaxie s, depend cr ucially on the galaxy mass 
(e.g. Haines et al. l2006al : [2007]) . and the strong bimodality 
in the properties of galaxies about a characteristic stellar 
mass of ~ 3 x 10 10 M Q (~ M*+l, Kauffmann et al. I2003D 
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Figure 1. The Shapley NIR survey (thick red) is shown superimposed to the surface density of R, < 21.0 galaxies as derived from the 
SOS (see text). Isodensity contours are shown at intervals of 0.25 galaxies arcmin -2 , with the thick contours corresponding to 0.5, 1.0 
and 1.5 galaxies arcmin" 2 , the densities used to separate the three cluster environments. The multi- wavelength photometric coverage of 
ACCESS on the SSC are also shown. Magenta: 24 (im, green: 70 fim, blue dashed: 150 nm and 250 nm. 



implies fundamental differences in the formation and evo- 
lution of giant a nd dw arf galaxies (e.g. Dekel & Birnboim 
200fJ; Keres et al. l2005h . This issue needs data-sets reaching 
much fainter luminosities than those of 2MASS to be inves- 
tigated in order to obtain, in general, an overall picture of 
galaxy evolution and, in particular, to establish the contribu- 
tion of the dwarf galaxy population to the total stellar mass 
in the local universe and the physical origin of the claimed 
faint-end upturn. The recent development of wide-field NIR 
imagers on 4-m class telescopes such as UKIRT/ WFC AM 
and KPNO /NEWFIRM has opened the possibility of NIR 
surveys to be > 100 x more sensitive covering many square 
degrees (e.g. UKIDSS), allowing the J\"-band LF of nearby 
clusters to be obtained covering not only the cluster cores, 
but the entire virialized regions. 

In this context we study the i^-band LF of the Shapley 
supercluster core (SSC) down to the dwarf regime (reaching 
~M*+6) with the aim of i) quantifying the environmental 
impact on the shape of the NIR LF; ii) deriving the stel- 
lar masses of the supercluster galaxies; iii) investigating the 
mechanisms driving galaxy evolution as function of galaxy 
mass. This work is carried out in the framework of the joint 
research programme ACCESS aimed at determining the im- 
portance of cluster assembly processes in driving the evolu- 
tion of galaxies as a function of galaxy mass and environment 
within the Shapley supercluster (see Sect. [2}. In Sect. [3] we 
describe the data-sets. In Sect. [4] we derive the NIR galaxy 



luminosity functions obtained through background subtrac- 
tion in the whole observed field and we study the ongoing 
effects of environment by comparing the LFs of galaxies in 
three different regions of the supercluster, characterized by 
high-, intermediate- and low-density, we also compare NIR 
and optical LFs. The galaxy stellar mass function is pre- 
sented in Sect. [5] The results are discussed in Sect. [6] and 
the summary and conclusions of this work is given in Sect. [7] 
Throughout the paper we adopt a cosmology with 
f2_M=0.3, S7a= 0.7, and Ho=70 km s _1 Mpc _1 . According to 
this cosmology 1 arcmin corresponds to 60 kpc at 2=0.048 
and the distance modulus is 36.66. 



2 THE ACCESS PROJECT 

ACCESS^ (A Complete Census of Star-formation and nu- 
clear activity in the Shapley supercluster, PI: P. Merluzzi) is 
a project whose aim is to distinguish among the mechanisms 
which drive galaxy evolution across different ranges of mass 
examining how, when and where the properties of galaxies 
are transformed by their interaction with the environment. 
Since the most dramatic effects of environment on galaxy 
evolution should occur in superclusters, where the infall and 
encounter velocities of galaxies are greatest (>1000 km s~ ), 

5 http://www.oacn.inaf.it/ACCESS 
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groups and clusters are still merging, and significant num- 
bers of galaxies will be encountering the dense intra-cluster 
medium (ICM) of the cluster environment for the first time, 
we choose to st udy t he core region of the Shapley super- 
cluster (Shaplev ll930h . The Shapley supercluster core is the 
richest and most dynamically active region in the local Uni- 
verse and hence represents a unique laboratory for study- 
ing the effects of the hierarchical assembly of structures on 
galaxy evolution. 

The multi-wavelength data-set available for this project 
includes panoramic imaging in UV [Galaxy Evolution Ex- 
plorer (GALEX); PI: R.J. Smith], optical [European South- 
ern Observatory (ESO) WideField Imager (WFI) archive 
data], near-infrared (UKIRT/WFCAM, PI: R.J. Smith), 
mid-infrared (Spitzer, PI: CP. Haines) all of which cover 
at least a 2 deg 2 area of the SSC (see Fig. 1). Fur- 
thermore, high-S /N medium-resolution optical spectroscopy 
(AAOmega, PI: R.J. Smith) was obtained for a sample 
of 5 41 galaxies in the SSC field (Smith, Lucey & Hud- 
son 120071 : [200^), of which 448 are supercluster members 
(0.039 < z < 0.056). For this sample 371 galaxies have B, 
R and K photometry. The spectroscopic sample is enlarged 
to ~650 galaxies with published data, NASA Extragalactic 
Database for bright objects, and 6dF data. New medium- 
resolution integral-field spectroscopy will be pr ovided by the 
Wide Field Spectrograph (WiFeS, Dopita et al. 120071 ) at the 
Australian National University 2.3 m telescope. A large pro- 
gramme of WiFeS observations (PI: M. Dopita & P. Mer- 
luzzi) started in April 2009. Finally, archive X-ray and ra- 
dio data are also available. The depth (e.g. B=22.5mag, 
_R=22mag, A"=18mag) and high-S/N of the data allows the 
investigation of the photometric and spectroscopic proper- 
ties of superclu ster g alaxies well int o the d warf regime (e.g. 
Mercurio et al. 120061 ; Gargiulo et al. l2009T ). 

The main scientific goals of ACCESS are: searching for 
the ram pressure effects; probing galaxy merging and galaxy 
harassment and "suffocation"; determining the frequency 
and the radial distribution of cluster AGN; obtaining a sta- 
tistical census of obscured star-formation in cluster galax- 
ies; correlate obscured star formation with hierarchical clus- 
ter assembly; compare mid-IR, optical, radio and UV star- 
formation indicators; investigating the fundamental plane of 
low mass early-type galaxies. Partners of this collaboration 
are the Universities of Durham and Birmingham (UK), the 
Italian National Institute of Astrophysics with the Observa- 
tory of Capodimonte and the Australian National University 
(FP7-PEOPLE-IRSES-2008, Grant agreement No. 230634). 



3 THE DATA 

The new near-infrared data analysed in this paper are com- 
plemented by panoramic B- and i?-band imagin g from the 
Shapley Op tical S urvey (SOS, Mercurio et al. 20061 and 
Haines et al. l2006bl . hereafter MMH06 and HMM06, respec- 
tively) . In Fig. [T] the optical and NIR survey are superim- 
posed. In the following we describe the two surveys and their 
data products. 



3.1 The Shapley Optical Survey 

The SOS comprises wide-field B- and _R-band imaging cov- 
ering a 2.0 deg 2 region towards the clusters A3562, A3558 
and A3556 which form the core of the Shapley supercluster 
at z~ 0.05. 

The observations were carried out with the Wide Field 
Imager (WFI) camera, a mosaic of eight 2046 x 4098 pixels 
CCDs giving a field of view of 34' x 33', and mounted on 
the Cassegrain focus of the 2.2m MPG/ESO telescope at La 
Silla. The survey is made up of eight contiguous fields, each 
with total exposure times of 1500 s in B band and 1200 s 
in R band, and typical FWHMs of 0.7-1.0 arcsec. The data 
were retrieved from the ESO archive and reduced using the 
ALAMBIC pipeline (version 1.0, Vandame liooi ) and cal- 
ibrated to the Johnson-Kron- C ousin s photometric system 
using observations of Landolt l|l992h standard stars. The 
sources are then extr acted and classified using SExtractor 
(Bertin & Arnouts I1996T I, resulting in galaxy catalogues 
which are both complete and reliable (i.e. free of stars) to 
_R=22.0mag and _B=22.5mag. Full description of the obser- 
vations, data reduction, and the production of the galaxy 
catalogues are described in MMH06. 



3.2 The A-band survey 

The A-band survey of the Shapley supercluster core was 
carried out at the United Kingdom Infra-Red Telescope 
(UKIRT) with the Wide Field Infrared Camera (WFCAM) 
in April 2007. The WFCAM instrument consists of four 2048 
x 2048 Rockwell detectors with a pixel scale of 0.4 arcsec. 
The four detectors are spaced by 94% of their active area. 
A single exposure covers an equivalent area of 0.19 deg 2 and 
four interleaved exposures are required to achieve a filled tile 
of 0.865 deg on a side (0.78 deg 2 ). We observed a mosaic of 
five complete tiles, covering a 3.043 deg 2 (of which ~2deg 2 
overlap with the SOS) region centred on the SSC, which 
comprises three Abell clusters A3556, A3558 and A3562 and 
two poor clusters SC 1327-312 and SC 1329-314, as shown 
in Fig. [1] The total exposure time for each field is 300 s, 
reaching A=19.5mag at 5cr, with typical FWHMs of 0.9- 
1.2 arcsec. 

The data were pipeline processed at WFCAM Sci- 
ence Archive ( WSA)/Cambridge Astronomy Survey Unit 
(CASU), reducing the frames and performing astrometric 
and photo metric calibration with respect to 2MASS (Irwin 
et al. l2004l ). Zero-point uncertainty is 0.0 15 mag and astrom- 
etry accuracy is <0.1 arcsec (Irwin et al. l2004h . 

For each frame, a photometric catalo gue w as derived 
by using SExtractor (Bertin & Arnouts [l996j) . We mea- 
sured magnitudes within a fixed aperture of 17 arcsec diam- 
eter, corresponding to ~8kpc at z~0.05, and Kron (Kron 
1980) magnitudes, for which we used an adaptive aperture 
with diameter a ■ iKron, where XKron is the Kron radius 
and a is a constant. We chose a = 2.5, yielding ~ 94% of 
the tota l sour ce flux within the adaptive aperture (Bertin & 
Arnouts 1 19961 '). We measured the Kron magnitude for all the 
objects in the catalogue and adopted it as the total mag- 
nitude. Luminosity functions were computed by means of 
Kron magnitudes, while aperture magnitudes were used for 
measuring galaxy colours. Since we derive galaxy colours us- 
ing the same apertures both at optical and NIR wavelengths, 
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we checked the effects of seeing variations among the wave- 
bands by degrading the J?-band image to the seeing of the 
if-band image. Comparing the aperture magnitudes in the 
original and the degraded image we find a difference which is 
an order of magnitude lower than the photometric error, as 
expected since the aperture is large compared to the seeing. 

Particular care is needed to avoid stellar contamination 
due to the high number density of both stars and galax- 
ies in this field (the Galactic latitude of this field is +30°) 
which increases the frequency of star-star and star-galaxy 
blends that can be misclassified as a single galaxy. For the 
star/galaxy classification we make use of the optical pho- 
tometry when available in the sense that objects observed in 
both R and if bands were classified as stars and galaxies ac- 
cording to the i?-band classificatior0. As shown in Tab. 3 of 
MMH06, at R = 20.0 - 20.5 mag, only 2% of the stars were 
misclassified as galaxies. According to the typical R — K 
colour of early-type galaxies at z~0.05 (Poggianti Il997fi . 
this magnitude range corresponds to if ~ 17.5 — 18.0 mag. 
The contamination of misclassified stars is taken into ac- 
count in the galaxy LF determination. For those objects 
observed in the if-band having no optical magnitude mea- 
surement, we use the distribution of sources in the stellarity 
index (SI) parameter of SExtractor versus Kron magni- 
tude (if) diagram to separate stars and galaxies. We clas- 
sified as stars those objects whose SI value is larger than a 
given threshold: SI m i n =0.8. This value of SI m i n has been cho- 
sen by adding simulated stars and galaxies to the if-band 
images, and measuring their SI and if parameters by means 
of SExtractor, in the same way as for real sources. Simu- 
lated stars and galaxies were randomly generated in a mag- 
nitude range of if =12-2 mag using the software 2DPHOT 
(La Barbera et al. |200S|) . 

The completeness of the if-band catalogue was esti- 
mated by measuring the percentage of simulated galaxies 
and stars which are recovered by SExtractor as a func- 
tion of K-band magnitude. The completeness function was 
found to be strongly dependent on the source density and 
therefore is different for each if-band frame. In fact, in high 
density regions the catalogue is ~ 65% and ~ 50% com- 
plete at if =17. 5 mag and if =18.0 mag, respectively, while 
in low density regions it is ~ 100% and ~ 80% complete 
at if =17. 5 mag and if =18.0 mag, respectively. We correct 
galaxy counts using a different completeness function for 
each if-band frame, i.e. by taking both the crowdedness and 
magnitude effects fully into account for each galaxy, when 
determining its contribution to the counts in a given magni- 
tude bin. By weighting each galaxy by the locally estimated 
completeness of the survey, we are able to obtain unbiased 
comparisons of the galaxy counts in different environments. 
Since in the high density region the completeness is less than 
50% beyond if =18.0 mag, we adopted this conservative limit 
as the magnitude to which catalogues can be reliably cor- 
rected for incompleteness when determining the galaxy LF. 
At this limiting magnitude the accuracy of the completeness 
functions is better by 5% and is taken into account in the 



6 The i?-band WFI mosaic is the deepest and highest resolution 
data obtained for this project, reaching M*+7, and is therefore 
generally used as reference for our multi- wavelengths surveys. 



Table 1. Central coordinates of DXS fields used for the statistical 
background subtraction. 



Name 


RA(J2000) 


DEC(J2000) 


XMM - LSS 


02 


25 


00 


-04 


30 


00 


ELAISN1 


16 


10 


00 


+54 


00 


00 


SA22 


22 


17 


00 


+00 


20 


00 





error budget of each galaxies. The final catalogues consist of 
18,534 galaxies with if ^18 mag. 



4 NIR LUMINOSITY FUNCTIONS 

The if -band galaxy LF of the SSC has been derived down 
to the magnitude limit if =18 mag accounting for interlopers 
by the statistical subtraction of the background contamina- 
tion. We chose this approach since we do not have spectro- 
scopic information complete for whole the galaxy sample in 
the considered magnitude range (being ~90% complete for 
R < 16 mag or if <13.5) and the available photometry (B, 
R and if band) does not allow us to derive accurate photo- 
metric redshifts. The number counts in the supercluster field 
has been obtained by weighting each galaxy's contribution 
to a given magnitude bin according to its completeness. We 
also correct for the contamination of the misclassified stars. 
Absolute magnitudes were determined using the k- and evo- 
lutionary c orrect ions for early-type galaxies at z ~0.05 from 
Poggianti (| 19971 ). The large observed area and the depth of 
the survey are suitable to investigate the effects of environ- 
ment within the supercluster. 



4.1 Background galaxy subtraction 

Since the area covered by the SSC observations lies com- 
pletely within the overdensity corresponding to the core 
complex, it is not possible to use the outer regions of the 
survey to estimate the background/foreground contribution 
to the galaxy counts. Therefore, we performed the statisti- 
cal subtraction of field galaxies by means of a control field 
observed with the same instrument at a suitable depth. A 
similar or larger area is necessary in order to reduce the 
effects of field-to-field variance and small number statistics. 

To this aim we chose the UKIRT Infrared Deep Sky Sur- 
vey ( UKID SS) Deep Extragalactic Survey (DXS) (Lawrence 
et al. l2007n which aims to map 35 deg 2 of sky to a magnitude 
limit of if =20.8 mag at 5a. Since our if -band photometry 
is 50% complete down to if =18.0 mag, the UKIDSS DXS 
data are suitable to estimate field galaxy counts. 

The background contamination was estimated from 
three control fields from the UKIDSS DXS 3rd data release 
(Warren et al., in preparation) reaching the required depth. 
In particular, we considered 24 multiframes of ~ 0.19 deg 2 
, covering a total area of ~4.55 deg 2 over three regions of 
sky (Tab.[T|, with exposure times of > 360 s in if. We note 
that this area is significantly larger than that covered by any 
published table of galaxy count s reac hing if = 18 or deeper, 
the survey s of Vaisanen et al. (|2000f ) and Kiimmel & Wag- 
ner (|2001 ) limited to areas of ~ 0.9 deg 2 , and reaching just 
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Figure 2. Comparison of galaxy counts obtained from the DXS 
(open triangles), CADIS (open circles) and ALHAMBRA (open 
squares). The counts are normalized to the total area covered by 
the CADIS data. 



K =17-17.5. The if -band catalogues were obta ined fr om the 
WFCAM Science Archive (WSA, Hambly et al. l200St ) select- 
ing from the dxsDetection table the isolated (pperrbits^Cl) 
objects with non-stellar morphologies (class=l). 

Since we have accounted statistically for the incomplete- 
ness affects in the K-band survey (see Sect. I3.2|l . we can 
subtract the control field counts from those obtained in the 
Shapley area in order to obtain superc luster member counts. 
In this case, following Bernstein et al. (|l995[ ) the background 
counts are estimated as the mean of the control field counts 
corrected for th e rati o between the observed areas (Eq. 1 of 
Bernstein et al. |l_995j) and errors on the background counts 
are estimated through an empirical approach as the rms of 
the counts in each control field with respect to the m ean es - 
timated over the whole area (Eq. 2 of Bernstein et al. ll995h . 
For the background counts, scaling the rms value to the 
whole area, we obtain an uncertainty on the mean counts 
of ~11% at K < 13.5 mag where small number statistics 
dominates, and less than 3% at if=18mag. For a Gaussian 
distribution, the standard deviation estimated from three 
samples is within a factor of two from the true value, in 
80% of trials. Since each field is smaller than the Tf-band 
survey, we overestimate the field-to-field variance among 
larger fields, taking into account that the fluctuations due 
to galaxy clustering are s malle r for wider fields. Following 
Ellis & Bland-Hawthorn (|2007l ) we estimate that the field- 
to-field variance is overestimated of about the 15%. Using 
this method the error on background counts accounts for 
the Poissonian fluctuations and the field-to-field variance. 
We also include the error related to the photometric uncer- 
tainties which is estimated by means of 100,000 Monte-Carlo 
simulations, changing the galaxy luminosities according to 
the photometric error and re-computing the number counts 
in each magnitude bin. 

These galaxy number counts were found to be con- 



sistent with field galaxy counts from the Calar Alto Deep 
Imaging Survey (CADIS, see Huang et al. 120031 ). a medium 
deep Jf-band survey with a total area of 0.2 deg 2 and a 
completeness magnitude of 19.75 mag (see Fig. [2}. We also 
agree with the counts of the Advanced Large Homogeneous 
Area Medium-Band Redshift Astronomical (ALH AMBRA) 
survey given by Cristobal- Hornillos et al. (2009]) covering 
a 0.44 deg 2 and reachi ng K g=19.5mag. We note that El- 
lis & Bland-Hawthorn (120071 ) who have combined numerous 
published galaxy number counts over a wide range of pass- 
bands and magnitudes, obtain galaxy counts in the range 
13 < K < 18 that are 10-15% higher than ours, but are 
based on small patches of sky (< ldeg 2 ) and show signifi- 
cant variations from one magnitude bin to another as sur- 
veys fall out of the sample. 

The counts of SSC galaxies were defined as the differ- 
ence between the counts detected in the supercluster fields 
and th ose e stimated for the background (Eq. 3 of Bernstein 
et al l 19951 ). Then, the uncertainties were measured as the 
sum in quadrature of fluctuations in the backgro und an d in 
the supercluster counts (Eq. 4 of Bernstein et al. 1 19951 ). 



4.2 The total luminosity function 

Figure [3] shows the Shapley LF over the whole Tf-band sur- 
vey covering the SSC (Fig. [TJ, and the fit with a single 
Schechter function. The error bars on number counts take 
into account Poissonian uncertainties, field-to-field variance 
(see Sect. 14. ip . photometric uncertainties and, in the last 
bin, the percentage of misclassified stars (see Sect. 13 . Sff) . 

The LF fit was obtained with a y 2 minimization rou- 
tine, accounting for the finite size of magnitude bins by in- 
tegrating the Schechter function over each magnitude bin. 
The Schechter function fit, whose parameters are the faint- 
end slope, a, the characteristic magnitude, M*, and the 
luminosity density 0, according to the \ 2 statistics, pro- 
vides an acceptable description of cluster galaxy counts. 
The best-fitting values of the Tf-band Schechter function 
parameters are M* K = -24.96 ± 0.10 and a = -1.42 ± 0.03. 
The confidence contours on a and m* are shown in the 
small panel in Fig. [3] and were derived by randomly shifting 
galaxy number counts according to their uncertainties, and 
re-computing the best-fitting Schechter function for each re- 
alization. The fit parameters and associated \ 2 statistics are 
listed in Tab. H 

The NIR LF trend derived in the 3 deg 2 area of the 
SSC agrees with the results of Mobasher & Trentham (1998) 
who found «=-1.41 in the magnitude range -19. 5< Mk <- 
16.5, although they studied only a 41.1arcmin 2 region of 
the Coma cluster core corresponding to about 10 arcmin 2 at 
Shapley redshift. 

In Fig. [ 5] we p lot for comparison the field if-band LF by 
Jones et al. (120061 ) who found M^ = -24.60 and a = -1.16. 
The two LFs appear consistent down to MJJ- = —23, but the 
faint-end slope for the SSC is steeper than that measured 
in the field at > 3cr level. The field value of M% is also 
not consistent with that obtained for the SSC with a single 
Schecter function, with MjJ- being brighter for the SSC than 
for the field, at the 3cr level according to the confidence 



tor the held, at the 6a level according 
contours in Fig. 10 of Jones et al. (2006). 
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K — band Magnitude 

Figure 3. K-band LF of the whole SSC and its fit with the Schechter function, contin uous line. The counts are per half magnitudes 
bins. The dot-dashed lines is the fit to the iC-band LF in the field of Jones et al. 2006 normalized to the luminosity density <j> of SSC 
LF. In the small panel the 1, 2 and 3cr confidence levels of the best-fit parameters for a (x-axis) and m* (y-axis) from the Schechter fit, 
are shown. 



Table 2. Fits to the LFs. Errors on the m* and a parameters can be obtained from the confidence contours shown in Figs. |31 and l5l 
Column 1: the analyzed region. Column 2: the fitting function. S for a single Schechter, G+S for Gaussian plus Schechter and S+S for 
two Schechter functions. Columns 3 and 4: characteristic magnitude of the Schechter function (apparent m* and absolute M*) or central 
magnitude fi of the Gaussian function. Column 5: the value of slope a or amplitude a of the Schechter and the Gaussian function, 
respectively. Columns 6, 7 and 8: the same of columns 3, 4 and 5 for the faint-end of the LF. Column 9 and 10: reduced x 2 an d its 
probability. 



Region Function | m* / M*/fl a/ a | mj Mjf otf | X? P(x 2 > X?) 
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96 


-1 
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29 


21 
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density 
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48 


-25 


18 


-1 


33 
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69 
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high 


density 
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13 


81 


-22 


85 
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97 


12.58 ±0.12 


-24.08 


-1.65 ±0.03 





58 


83 


46% 


high 


density 
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11 


57 


-25 


09 


-1 


22 


16.64 ±0.29 


-20.02 


-2.05 ±0.50 
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47 


-25 


19 


-1 


44 








1 


30 


20 


39% 
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G 
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13 


81 


-22 


85 


1 


90 


12.47 ±0.11 


-24.19 


-1.64 ±0.02 


1 


41 


15 


31% 


int 


density 


S 


+ 


s 


11 


98 


-24 


68 


-1 


26 


14.44 ±0.21 


-22.22 


-2.41 ±0.11 


1 


34 


20 


24% 


low 


density 




s 




11 


51 


-25 


15 


-1 


49 








1 


00 


44 


78% 


low 


density 


G 


+ 


s 


13 


79 


-22 


87 


1 


58 


14.13 ± 0.11 


-22.52 


-1.74 ±0.04 





20 


99 


60% 


low 


density 


S 


+ 


s 


12 


50 


-24 


16 


-1 


04 


15.72 ± 0.14 


-20.94 


-1.90 ±0.12 





18 


99 


77% 



4.3 The effect of environment 



Balogh et al. (|200lh . using 2MASS data, were the first to 
detect environmental variances in the NIR LF. T his env iron- 
mental effect was then questioned by Rines et al. (2004|) who 
found that the cluster LFs derived in the virial regions and 
in the infall regions are very similar, although both poorly 
fitted by a Schechter function. However, in their work, which 
is also based on the 2MASS survey, they noted that at mag- 
nitudes fainter than the completeness limit the LF in the 



infall regions may indicate a steeper faint-end slope which 
should be investigated by means of deeper data-sets. 

In order to investigate the effects of the environment we 
derived and compared the LFs in three different regions of 
the supercluster, characterised by high-, intermediate- and 
low-densities of galaxies (see Fig. [TJ where galaxy densities 
are p >1.5, 1.0< p and 0.5< p ^1.0 gals arcmin -2 , 

respectively. The local density of R < 21 mag galaxies, E, 
was determined across the i?-band WFI mosaic (i.e. in a 2 
deg 2 area, see Fig. 1). We derive £ by using an adaptive 
kernel estimator (Pisani ll993l ; ll996l ). in which each galaxy i 
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K — band Magnitude 

Figure 4. The _R"-band LFs of galaxies in the three cluster regions corresponding to high- (rilled circles), intermediate- (open circles) 
and low-density (crosses) environments. In the left, central and right panels the continuous, long-dashed and short-dashed lines represent 
the fits with Schechter, G+S and S+S functions to high-, intermediate- and low-density regions, respectively. The counts are per half 
magnitude bin. 




a 

Figure 5. The 1, 2 and 3<r confidence regions for the i\-band 
Schechter parameters (right panel in Fig. |4j for the three clus- 
ter regions corresponding to high- (solid contours), intermediate- 
dotted) and low-density (dashed) environments. 



is represented by a Gaussian kernel, K(ri) oc exp(— r 2 /2of ), 

— 1/2 

whose width Oi is proportional to £ i thus matching the 
resolution locally to the density (see MMH06 for more de- 
tails) . 

In left-hand panel in Fig. [3] left panel, we show 
the if-band LFs of galaxies in the high- (filled circles), 
intermediate- (open circles) and low-density (crosses) re- 
gions covering areas (in the SOS/K -band survey overlap) 
of ~0.115, 0.330 and 1.062 deg 2 , respectively, together with 
their fits with a Schechter function (continuous, long-dashed 
and short-dashed lines, respectively). The background sub- 
traction was performed as for the total LF simply scaling 
the counts with the area values because of the complex ge- 
ometry of the three density regions. Figure [S] shows the con- 
fidence contours of the best fitting Schechter function for the 
three density regions. The faint-end slope becomes steeper 
from high- to low-density environments varying from -1.33 
to -1.49, being inconsistent at the 2a confidence level (c.l.) 
between high- and low-density regions. We note that in this 



investigation the results are also more robust than those 
obtained for the total LF against variations in background 
galaxy counts, since we are considering denser superclus- 
ter regions and, furthermore the control field is significantly 
larger than the regions of different environments. 

This result seems to be in contradiction with p revious 
works (e.g. Balogh et al. l200ll ; Croton et al. 120051 ) which 
found a LF trend that varies smoothly with local density 
and/or environment. It should be note that these studies 
are based on the 2MASS survey which allows to investigate 
the luminosity distribution only down to about M*+2 at 
the redshift of Shapley, while we are considering a much 
fainter galaxy population which is more likely af fected by 
environmental related processes (e.g. Haines et al. 120071 ). 

It is also worth pointing out that comparing the flat 
NIR LF measured for field gala xies b y recent surveys (e.g. 
6dF Galaxy Survey, Jones et al. 120061 ) with our slope in the 
low-density environment, one has to take into account that, 
since we are still in the cluster environment, this region can 
be suitably associated to the infall region rather than to the 
field. On the other hand, the LF slopes obtained by C AIRNS 
(Cluster And Infall Region Nearby Survey; Rinesetal. l2004l ) 
in both cluster virial and infall regions are consistent with 
the value derived for the overall SSC with a single Schechter 
function. 

According to the \ 2 statistics, in all the three environ- 
ments the fit with a single Schechter function cannot be re- 
jected, but there is some "structure" evident in the residuals: 
the fit systematically under- and over-predicts the observed 
counts as a function of magnitude. The LFs suggest instead 
a bimodal behaviour due to the presence of an upturn for 
faint galaxies, that cannot be described by using a single 
Schechter function. To successfully model these changes in 
slope and to compare our results with our optical LFs (see 
Sect. 14. 4[) , we fit our data with a composite Gaussian + 
Schechter (G+S) LF (Fig. [4] central panel) and the sum of 
two Schechter (S+S) functions (Fig. 21 right panel). Look- 
ing at Fig.g]and Tab.[2]we note that the G+S and S+S fits 
significantly improve the data description, particularly in 
the low-density region, where according the reduced \ 2 the 
probability of the fit is P(x 2 > xl) ~ 99-6% and 99.8% for 
the G+S and S+S functions against P(x 2 > xl) ~ 46% for 



NIR Luminosity and Stellar Mass Functions 9 



Absolute Magnitude 




Apparent Magnitude 

Figure 6. Galaxies LFs B- (left panel), i?-band (central panel) and A"-band (right panel) high- (filled circles), intermediate- (open 
circles) and low-density (crosses) environments. The long-dashed and short-dashed lines represent the the G+S best fit for intermediate- 
and low-density environments respectively and the continuous line is the Schechter best fit for high-density regions. 



the single Schechter function. Since the function fitting the 
faint-end is poorly constrained by the data (the faint com- 
ponent dominates only in the last three magnitude bins), in 
order to estimate the uncertainties on the parameters char- 
acterizing the LF at fainter luminosities, mf and at, we pro- 
ceed as follows. Fixing all the best fit parameters except mf 
or ctf, we randomly shift galaxy number counts according 
to their uncertainties, and then re-compute the best-fitting 
functions obtaining a range of values for mj and at. The 
faint-end slope becomes steeper from high- /intermediate- to 
low-density environments varying from -1.65 to -1.74 with 
the G+S fit. 

4.4 Comparison with Optical LFs 

In order to investigate the processes responsible for changing 
the shape of the galaxy LF, we compare the trends observed 
for the optical and the NIR LFs. Since the NIR LF is ex- 
pected to approximate the SMF, while the optical LFs are 
more sensitive to the galaxy star-formation history, both are 
needed for investigating the nature of galaxies which domi- 
nate the faint-end. For instance, a steep optical LF can be 
compatible with a flat stellar mass function if dwarf galax- 
ies have their luminosities boosted by starbursts, a scenario 
which can be probed by comparison to the NIR LF. 

The optical LF of the SSC derived in the SOS (see 
MMH06) cannot be described by a single Schechter func- 
tion due to the dips apparent at M*+2 both in B and R 
bands and the clear upturn in the counts for galaxies fainter 
than B and 7? ~18 mag. Instead the sum of a Gaussian and 
a Schechter function, for bright and faint galaxies, respec- 
tively, is a suitable representation of the data. Furthermore, 
we observed significant environmental trends in the form of 
a dip which becomes deeper, and a faint-end slope which 
becomes steeper, with decreasing density. In particular, the 
slope values becomes significantly steeper from high- to low- 
density environments varying from -1.46 to -1.66 in B band 
and from -1.30 to -1.80 in R band, being inconsistent at 
more than 3cr c.l. in both bands. Such a marked luminos- 
ity segregation is related to the behaviour of the red galaxy 
population: while red sequence counts are very similar to 
those obtained for the global galaxy population, the blue 
galaxy LFs are well described by single Schechter functions 



and do not vary with the density. We explained these re- 
sults in terms of the galaxy harassment scenario, in which 
the late-type spirals that represent the dominant popula- 
tion at ~M*+2 are transformed by galaxy harassment into 
passively-evolving dwarf spheroids, and in the process be- 
come ~l-2 magnitudes fainter due to mass loss and an age- 
ing stellar population without new star formation. The ob- 
served changes in the shape of the LF can be considered as 
reflecting the changes in the mixture of galaxy morpholog- 
ical types with environment described by the morphology- 
density relation, from late- type dominant (and hence a steep 
faint-end to the LF) in low-density regions to early-type 
dominant (with a shallower faint-end to th e LF) , in the clus- 
ter cores (Binggeli, Sandage & Tammann Il988l ; de Lappar- 
ent l2003h . 

In Fig. HI] we compare the shape of the composite G+S 
B (left panel), R (central panel) LFs, derived in MMH06, 
with the relative K-b&nd LFs (right panel) in the three den- 
sity regions. Both LFs (optical and NIR) are steeper in the 
low-density regions. However, the NIR LFs only marginally 
indicate an absence of M*+2 galaxies in low-density envi- 
ronment. 

The study of the optical LFs showed that it is the red 
galaxy populations that turn out to contribute to the change 
of the LF trend with environment, in particular to the steep- 
ening of the faint-end slope. This is at odds with what is 
found in the field where the faint-end is populated by galax- 
ies of smaller masses, later morphologies, bluer colours, later 
spect ral types, and str onger line emission (e.g. Balogh et al. 
l200ll ; Madgwick et al. 120021 ). On the other hand, in clusters 
the LF of early-type galaxi es is f ound to be steeper than in 
the field (De Propris et al. 120031 ). We should also take into 
account that at least part of the red galaxies contributing the 
Shapley LFs in different s upercl uster environments may not 
be passive: Haines et al. l|2008h studying a volume-limited 
sample (0.005< z <0.037) of local galaxies found that ~30% 
of red sequence galaxies in the optical colour-magnitude dia- 
gram show signs of ongoing star-formation from their spec- 
tra and this contamination is greater at faint magnitudes 
(M r >-19mag). 
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5 THE GALAXY STELLAR MASS FUNCTION 

The combined optical and NIR data allow us to derive the 
distribution of galaxy stellar masses. The sample we anal- 
ysed is in the magnitude range 10^ K ^18 and refers to 
the ~2 deg 2 area covered by both the SOS and our if-band 
imaging. We note that although the area of the NIR sur- 
vey is slightly different from that of the SOS, both surveys 
map the same kinds of environment from the low- to the Z 
high-density (see the isodensity contours in Fig. [1} . 

One of the main concerns in deriving the stellar mass 
function of SSC galaxies is the foreground/background con- 
tamination that has to be estimated and corrected for. There 
exists already a wealth of spectroscopic data in the region 
covered by optical and NIR photometry, comprising about 
650 galaxy redshifts (see Sect. [2} corresponding to ~ 90 
% of R < 16 mag galaxies. In order to extend the magni- 
tude range and improve the statistics, we need to adopt a 
complementary approach for those galaxies without spec- 
troscopic information. We use the probability that galaxies 
are supercluster m ember s as derived by HMM06 following 
Kodama & Bower (|200ll ). We consider separately the three 
cluster environments as well as the remaining galaxies in the 
SOS, and construct two-dimensional histograms with bins 
of width 0.4 mag in R and 0.2 mag in B — R to properly 
map the galaxy colour-magnitude distribution. The number 
counts in each bin are then compared with those expected 
for a suitable field region, normalized to have the same over- 
all area. For this purpose we used a 4.3 deg 2 region of deep 
BVR imaging from the Deep Lens Survey (Wittman et al. 
2002). The probability that a randomly selected SOS galaxy 
belongs to the supercluster is then determined from the ra- 
tio of the number counts obtained for that bin from the SOS 
and DLS (Eq. 1 of HMM06). For those galaxies with avail- 
able redshifts the probability is set to 1 for 0.035<2<0.05ff] 
or otherwise. 

The stellar masses of galaxies belonging to the Shapley 
supercluster, according to the previous criteria, contribute 
to the galaxy stellar mass function according to their likeli- 
hood of belonging to the Shapley supercluster. 

5.1 Derivation of stellar masses 

The stellar masses of galaxies belonging to the Shapley su- 
percluster are estimated by means of stellar population mod- 
els constrained by the observed optical and infrared colours. 
It is well known that stellar masses estimated using the fit 
to the multicolour spectral energy distribution (SED) are 
model dependent and subject to various degeneracies. In or- 
der to reduce such degeneracies we have used a larg e grid 
of complex stellar population models by Maraston (2005) 
with a Salpeter initial mass function covering a wide range 
of parameters. We use SEDs with the star-formation history 
(SFH) parameterised as oc exp(— t/r), with r between 
1.0 and 20.0 Gyr, ages between 0.001 and 14 Gyr and metal- 
licities in the range 0.5-2.0 Zq. We note that the met allicit y 
of low-mass galaxies can be lower (e.g. Smith et al. [2009), 
but at present complex stellar population models do not ex- 
plore such a low metallicity range. The synthetic spectra are 

7 The redshift range is derived from the redshift distribution of 
galaxies in the SOS field with available spectroscopy. 
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Figure 7. Stellar Mass Function of galaxies in the SSC (see 
text). The continuous line indicates the best single Schechter fit to 
the data. In the small panel the 1, 2 and 3<r c.l.s of the best-fitting 
parameters for a (x-axis) and logio M* (y-axis) are shown. 

shifted to the galaxy spectroscopic redshift, if known, or at 
the median supercluster redshift 2=0.05. Then for each of 
them we compu te the B, R and K magnitudes by adopt- 
ing the Calzetti < |200lf ) extinction. Since in the photometric 
calibration of the B band the colour term (B — R) is not 
negligible, as discussed in MMH06, we use the WFI B filter 
in order to compute B magnitude^] from the models. 

The most appropriate evolutionary history is selected 
by fitting the optical + NIR photometry, and the mass 
is estimated by normalising the best-fit SED to the ob- 
served if-band magnitude. This choice of model grid pa- 
rameters yields a fairly uniform coverage of colour space 
and well represents the SEDs of the galaxies in the sample. 
The uncertainty on the resultant stellar mass was estimated 
by performing Monte Carlo simulations, shifting the galaxy 
colours according to their corresponding uncertainties, and 
re-computing each time the mass. Further sources of un- 
certainty in the fitting technique are the error on redshift, 
which is fixed at 2=0.05 for those galaxies without avail- 
able spectroscopic redshift. We perform Monte-Carlo sim- 
ulations by randomly shifting SEDs in the redshift range 
0.035< 2 <0.056 adopted to select spectroscopic confirmed 
supercluster members. This contribution to the mass uncer- 
tainty is negligible, since it is at least one order on magni- 
tude lower than that due to the photometric errors. Besides 
of this, the main so urce of uncertaint y is t he adopted IMF 
(see Bell & de Jong 1200 if ). Bell et al. (120031 ) found that, us- 
ing the simple stellar populations with different IMF, stellar 
masses can be systematically increased by ~0.1 dex or de- 
creased by ~0.45 dex, thus resulting in a overall rescaling of 
the stellar mass. A zero-point shift to the stellar mass scale 

8 Our B-band photometry agrees within the zero-point uncer- 
tainties (~0.03 mag) with that obtained by the Wide -field Nearby 
Galaxy cluster Survey (WINGS; Varela et al. l2009t) using inde- 
pendent WFI B-band observations for a subset of the SOS field. 
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Figure 9. The 1, 2 and 3cr confidence levels for the Galaxy Stellar Mass Function parameters in the three cluster regions corresponding 
to high- (solid contours), intermediate- (dotted) and low-density (dashed) environments. 



would not affect any of our environment analyses, which are 
explicitly differential. 

The average error on the mass evaluation is ~20% and 
turns out to be 10-15% and 30-35% for galaxies with proba- 
bilities greater and less than 0.5 of being supercluster mem- 
bers, respectively. 

5.2 Galaxy Stellar Mass Function 

Figure [7] shows the stellar mass function of galaxies with 
1O 8 75 M < M < 1O 12 M . Based on our A"-band com- 
pleteness limit of K — 18, we conservatively derive the SMF 
down to .M = 10 8 ' 75 Mq, which corresponds to the estimated 
stellar mass of the quiescent galaxy population at our K- 
band limit. Fitting a single Schechter function, the recov- 
ered parameters are logio(.M*) = 11.16 and cv=-1.20 whose 
uncertainties are given by the confidence contours in Fig. [7] 
We check the robustness of our result against possible 
completeness issues of the galaxy sample used in deriving 
the SMF. According to their location in the R — K vs. K 
colour-magnitude diagram, only 5% of the galaxies in the 
K band catalogue which are possible supercluster members 
(i.e. which lie along or below the red sequence) do not have 
available estimates of probability. These galaxies are uni- 



formly distributed in the magnitude bins, and mostly lack 
optical data (and hence probabilities) due to their location 
near bright stars, which produce large ghosts in the optical 
data. We note also that the low-mass galaxies we may have 
lost would contribute to increase the low-mass-end slope of 
the SMF. 

For what concerns the observed trend of the galaxy 
SMF, the slope is in agreement with previous w orks c on- 
cerning the SMF of field galaxies (Cole et al. | 200ll . in 
the mass range 10 9 -10 n ' 5 M ; Panter et al. |2004 in the 
mass range 10 7 ' 5 -10 12 M©). On the other hand the faint- 
end slope of the SSC turns out t o be shallower than the 
value given by Balogh et al. ((2001) for the local mass func- 
tion of cluster galaxies [2MASS/ Las Campanas Redshift 
Survey (LCRS)] although the latter is affected by a large 
uncertainty (Aq ~0.1 at la). The value of M* is hi gher 
than that observed in the field (e.g. Bell et al. l2003h . as 
expected for cluster environment. The SMF of supercluster 
galaxies is characterised by an excess of massive galaxies 
that is associated to the cluster BCGs. 

In Fig. |8]we show the SMF for the different supercluster 
environments. Unlikely in the case of the LF no environmen- 
tal trend is seen in the slope of the SMFs (see Fig. [9} . On 
the other hand, the A4* increase from low- to high-density 
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regions and the excess of high-mass galaxies remains depen- 
dent on the environment. We remark that the uncertainty 
due to the adopted IMF (see Sect. 15. 1[) does not affect our 
analysis since the stellar masses at a fixed epoch will be all 
changed by the same amount. 



6 DISCUSSION 

6.1 The faint-end upturn in the i\-band LFs 

We observed a steep faint-end slope for the Shapley super- 
cluster K-band LF (Fig. [3| obtaining q=-1.42, similarly to 
values observed for the cluste r LFs of Mobasher & Tre ntham 
(|1998T ) and Andreon & Pello |200d ). De Propris et al. (Il998l ) 
observed a steeper slope of a=— 1.7 deriving the NIR LF for a 
wider area (29'. 2x22'. 5) of the Coma cluster (corresponding 
to ~15'xll' at z~0.05) down to # = 16 mag (K ~15.7), but 
the authors conservatively consider this result as an indica- 
tion of a steep LF for dwarf galaxies rather than a precise 
estimate of the slope. On the other hand, other studies found 
a less steep or flat faint -end slope in the NIR LF for cluster 
galaxies (e.g. Lin et al. |2004| ). 

The steep slope of the cluster LF has been discussed 
consi dering the otherwise flat LF of field galaxies (e.g. Cole 
et al. l200lf ) and how the envir onme ntal effects could shape 
the g alaxy LF. Balogh et al. l|200ll ; see also Croton et al. 
120051 ) demonstrated that there is a statistical difference be- 
tween the cluster and field galaxy NIR LFs, with a brighter 
M* and a steeper faint-end slope for clusters with respect to 
the field. An interpretation can be inferred from the trends 
observed for the L Fs of early- and late-type field galaxies. 
Huang et al. (|2003l ) found that the later type galaxies have 
a fainter M* and a steeper slope with respect to the early- 
type galaxies in the field. Their faint-end slope for late-type 
galaxies is equal to the value we obtained for the SSC (a =- 
1.42), although for a brighter sample. This suggests a sce- 
nario whereby the faint-end upturn in the cluster LF is due 
to late-type objects accreted from the field. This scenario 
is furt her supported by the result of Harsono & De Propris 
who did not detect the up-turn in two intermediate 
redshift clusters indicating that this feature has recent ori- 
gins. 

By considering the LFs of the Shapley supercluster in 
different density environments, we found a steepening of the 
faint-end slope which changes from a=-1.33 to «=-1.49, 
being inconsistent at 2cr c.l. between high- and low-density 
regions. Moreover, the general shape of the LFs in the low- 
density region turns out to be better reproduced using the 
combination of G+S functions. This observed bimodality in 
the LF and its variation with environment suggest a scenario 
where bright and faint galaxy populations have followed dif- 
ferent evolution histories and indicate that an environmental 
effect such as galaxy harassment and/or ram pressure strip- 
ping could be responsible for shaping the LF. This entails 
that the environment is responsible of the final mixture of 
the galaxy types, in particular for the faint/low-mass galax- 
ies. 

The stellar and/or field contamination can artificially 
produce the faint-end upturn. We checked carefully the is- 
sues of the stellar contamination (see Sec. l3.2[l and we chose 
a control field characterized by a deeper limiting magnitude 



and a larger area respect to the Shapley A'-band survey in 
order to rely on the star/galaxy classification and to ac- 
count for the field-to-field variance. Assuming the trend we 
observed to be real, we make a simple exercise to under- 
stand how the survey depth can affect the measured shape 
of the LF. In Fig.[lO]we show that the slope of the faint-end 
becomes clearly steeper as the depth of the sample is in- 
creased in all the supercluster environments, demonstrating 
the need for such deep data-sets to understand the role of 
environment on galaxy evolution. 

The observed trend with environment is more dramatic 
for the Shapley optical LFs obtained by MMH06. In particu- 
lar, the faint-end slope becomes steeper at > 3cr significance 
level from high- (ob =-1.46, or =-1.30) to low-density en- 
vironments (qb =-1.66, or =-1.80) in B and R bands. Also 
the bimodality of the gal axy LF , commonly obs erved for rich 
clusters (e.g., Yagi et al. 120021 ; Mercurio et al. I200ST ), turns 
out to be more evident in the optical bands. We note that 
the SOS is about 1.5 magnitude deeper in R band respect 
to the K band considering R — _K"=2.6. 

6.2 The galaxy stellar mass function 

The stellar mass functions derived for the supercluster galax- 
ies are in ge neral agreement with those obtained by Baldry 
et al. (l2006f ) who using SDSS data did not find changes in 
the slope of the SMF with environment, except for changes 
in the characteristic mass which increases with the local den- 
sity. We do find a stee per slo pe: «=-1.20±0.02, instead of 
a ~-l of Baldry et al. (120061 ) who did not quote error esti- 
mates. This discrepancy might be due to systematic uncer- 
tainties in the mass deri vation : in fact, using the MOPED 
algorithm, Panter et al. (2004]) obtained for the SDSS data 
a=-1.159±0.008 which is consistent within the errors with 
our findings. Another source of discrepancy can be due to 
the use of the if-band data to estimate the galaxy stel- 
lar masses, but the slope obtained by Bell et al. ()2003h 
combining 2MASS and SDSS data-sets is significantly lower 
(a=-0.86) respect t o tho se of SSC in all environments. As 
shown by Bell et al. (|2003r i the difference here may be due to 
the fact that 2MASS misses low surface brightness galaxies. 
They estimated the i^-band flux and stellar masses from the 
optical photometry for the galaxies not detected in AT-band 
and produced a p-band derived stellar mass function with 
q= -1.10. 

iBaldrv et al.l l|2008h found a strong low-mass upturn be- 
low M = 10 9 M Q analysing the NYU-VAGC0, their SMF 
being fitted by a S+S function with, in the low-mass end, 
a=-1.58±0.02. An upturn in the SMF was also shown by 
Jenkins et al. (2007) for the Coma cluster by means of 
Spitzer InfraRed Array Camera (IRAC) observations. They 
observed a steep increase in two different Coma regions be- 
low M — 1O 8,2 M with a ~-2 (but they did not quantify 
the trend with a fit of the data). 

The fact that we do not observe any upturn in the 
SMF of the Shapley supercluster can be related to the dif- 
fere nt ma ss ra nge or to environmen tal differences. Jenkins et 
al. (120071 ) and IBaldrv et all (|2008t) extend their analysis to 

9 New York Uni versity Value-Added Galaxy Catalog (Blanton et 
al. l2005al , l2005bh 
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Figure 10. Values of the LF slope a as function of the different magnitude cuts in high- (continuous line and filled circles), intermediate- 
(long dashed line and open circles) and low-density (short dashed line and open circles) environments. 



_A/f = 10 7 ' 5 M Q and X=1O 8 M , respectively, with the faint- 
end upturn becoming evident at M <1O 9 M0, very close to 
our mass limit is .M=1O 8 ' 75 M0. This suggest that the rea- 
son why we see an upturn in the Jf-band LF, but not in 
the SMF, is that the galaxies which cause the faint-end up- 
turn are star-forming galaxies with low stellar mass-to-light 
ratios and stellar masses in the range 10 8 < M <10 9 Mq. 
This would be consistent with Bolzonella et al. (|2009l ) who 
found that the upturn was due to late-type galaxies, but in- 
consistent with the finding of MMH06 in which the upturn 
appeared due to re d seq uence galaxies. 

Jenkins et al. (|2007T ) analysed two regions of the Coma 
cluster: one in the cluster centre (0.733 deg 2 corresponding 
to ~2.1 Mpc 2 ) and one off-centre region (0.555 deg 2 corre- 
sponding to ~1.6Mpc 2 ) located ~1.7 Mpc southwest. The 
area of the SSC covered both by the SOS and the 7^-band 
survey is ~30Mpc 2 . Therefore, we are studying a much 
larger area that comprises infall regions as well as cluster 
cores. Dynamical analysis indicates that at least a region 
of radius 11 Mpc centred on the central cluster A 3558, and 
possibly the entire superclus ter, is past turnaround and is 
collapsing (Reisenegger et al.[2000), while the core complex 
itself is in the final stages of collapse, with infall velocities 
reaching ~2000kms _1 . This differences in the environments 
may be responsible of differences in the stellar mass function 
for the two cosmic structures. Another, possible issue can be 
relate d to the background subtraction (see Rines & Geller 
120081 ). 

Investigating the environmental effect on the SMF in 
the SSC, we find that both M* and the excess of galax- 
ies at the bright-end increase as foreseen by the hierarchical 
models where the most massive galaxies formed in the den- 
sity peaks. On the other hand the faint-end slope does not 
change in the different supercluster environments suggest- 
ing that the mechanism that is acting in shaping the LF 
(see Sect. I4.3(l does not significantly affect the galaxy stellar 
masses. 



6.3 Comparison with the simulations 

We should expect changes in the A'-band luminosity func- 
tion with environment to reflect two processes: (i) the effect 
of the diverse merging histories of galaxies in different en- 
vironments within the context of the hierarchical merging 
scenario, in which cluster galaxies are likely to have formed 
earlier and had a more active merger history than field galax- 
ies that form in the smoother low-density regions; and (ii) 
the later impact of environmental processes such as tidal 
stripping, which may drastically reduce the stellar mass of 
galaxies in high-density regions. While it is very difficult 
to quantify the latter's contribution to the galaxy luminos- 
ity/mass function, we can attempt to measure the former 
contribution by comparison to the cosmological numerical 
simulations. 

To this aim, we extracted galaxy catalogues in the vicin- 
ity of the 20 most massive dark m atter h alos from the Mil- 
lennium simulation (Springel et al. l2005t ), corresponding to 
galaxy clusters with masses 7-23 x 1O 14 M0 and velocity dis- 
persions (800 < a < 1400 kms" 1 ). These simulations cover a 
(500/i~ 1 Mpc) 3 volume, producing DM halo and galaxy cat- 
alogue s base d on the semi-an alytic models (S AMs) of Bower 
et al. (200(J) and Font et al. (2008) for which positions, pe- 
culiar velocities, absolute magnitudes and halo masses are 
all provided at 63 snapshots to z = 0, allowing the orbit 
of each galaxy with respect to the cluster centre to be fol- 
lowed. We select member galaxies from these twenty clus- 
ters that have Mk < —17.5, and lie within 5 Mpc of the 
cluster centre, and show in Fig.[TT]the stacked K-band clus- 
ter luminosity functions of galaxies as a function of pro- 
jected cluster-centric distance, scaled by the rsoo value of 
each cluster's DM halo . We show the LFs obtained from 
both the Bower et al. [2006] (thick lines) and Font et al. 
120081 (thin lines) SAM galaxy catalogues, and the four dif- 
ferent curves correspond to r < 0.5 rsoo, 0.5 < r < 1.0 rsoo, 
1.0 < r < 2.0r5oo and r > 2.0 rsoo- For comparison, the rsoo 
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: of 0% that the observed and SAM LFs are drawn by the 
: same parent distribution. 

Although the merging history of galaxies at the faint- 
end (Mk > — 20) may not be fully resolved by the semi- 
analytic models, we may expect at least their stellar masses 
: (and hence i f-ban d luminosities) to be reasonably robust 
; (Bower et al. .200$). In previous works discussing the pres- 
\ ence of a dip in the galaxy luminosity function, it has been 
suggested the merging of L* galaxies as a possible pathway 
to form the dip. However the consistency of the luminos- 
ity functions with cluster-centric radius, suggests that the 
: variations observed in the Shapley supercluster (Fig. 6) are 
not primarily due to variations in the merging histories of 
galaxies, this being a process that should be well described 
by the semi-analytic models. Instead processes such as tidal 
stripping or harassment of infalling spiral galaxies, which 
. are not yet included in the semi-analytic models seem more 
: plausible pathways to produce the observed variations in the 
galaxy LFs. 



-27 -26 -25 -24 -23 -22 -21 -20 -19 -18 

Figure 11. Composite i-T-band LFs for the 20 most massive clus- 
ters in the Millennium simu lation, based on the semi-an alytic 
models of Bower et al. |2006| (thick lines) and Font et al. |2008| 
(thin lines), in bins of projected cluster-centric radius. The four 
different curves correspond to r < 0.5 7*500 (solid lines), 0.5 < r < 
1.0r5oo (dashed lines), 1.0 < r < 2.0 7*500 (dot-dashed lines) and 
r > 2.0r5oo (dot-dot-dashed lines). 



values for the clusters in the SSC derive d from Chan dra X - 
ray observations (Sanderson et al. 2006; Haines et al. 2009) 
are 1.29 Mpc (22.9 arcmin at z = 0.048) for A 3558, 0.91 Mpc 
(16.1 arcmin) for A3562 and SC 1327-312 and 0.76 Mpc 
(13.5 arcmin) for SC 1329-317. By inspection of Fig. 1, our 
high-, intermediate- and low-density regions correspond ap- 
proximately to the r < 0.5 7*500, 0.5 < r < 1.0 7-500 and 
1.0 < r < 2.0 7*500 radial bins respectively. 

It is immediately apparent from Fig. [TT] that there is 
little if any environmental dependence of the Tf-band lumi- 
nosity function, except at the very bright end (Mk < —26), 
while equally there are no significant differences between 
the two semi-analytic models. This is confirmed if we com- 
pare the derived best-fitting single Schechter functions to the 
SAM LFs, with M* K = -24.42±0.04, a = -1.134±0.007 for 
r < 0.57*500 and M* K = -24.37 ± 0.03, a = -1.167 ± 0.003 
for r > 2.0 7*500- Similarly, if we select just those galaxies 
which have already passed through the cluster core (based 
on following their orbits), and hence are most likely to cor- 
respond to the red sequence population, we find no signifi- 
cant variation in the LFs with cluster-centric radius. There 
is also no sign of an upturn at faint magnitudes, although 
we note the SAM galaxy catalogues start to become incom- 
plete at Mk> — 20 due to the limited mass resolution in the 
Millennium simulation (Bower et al. I2006T ) . The only clear 
difference among the LFs is the excess at the extreme bright 
end (Mk < K* — 2) in the inner cluster region, due to the 
BCG population. Moreover, the x 2 test gives a probability 



1 7 SUMMARY AND CONCLUSIONS 

It is well known that the NIR luminosities provide a more 
reliable estimate of the stellar masses compared to the opti- 
cal ones due to the fact that the mass-to-light ratios in the 
NIR vary by at most a factor 2 acr oss a wide range of star 
formation histories (Bell & de Jong l200ll) in comparison to 
the much larger variations of the M/L ratios (up to a factor 
10) observed at optical wavelengths. In addition the effects of 
the extinction are much weaker at infrared wavelengths that 
in the optical ones, and k-corrections for infrared colours are 
only weakly dependent on t he Hu bble type and vary slowly 
with redshift (e.g. Poggianti 1997). In our study we exploit 
new deep (K =18 mag) if-band imaging of the SSC comple- 
mented by the deep optical imaging down to _B=22.5mag 
and i?=22mag from the SOS, and a spectroscopically con- 
firmed supercluster sample of ~650 galaxies across the same 
field which is ~90% complete at R < 16 mag. We present an 
analysis of the A"-band LF of galaxies as a function environ- 
ment and we derive the galaxy SMF in order to constrain 
the physical mechanisms that transform galaxies in different 
environments as function of galaxy mass. 
Our results are summarized as follows. 

- The K-b&nd LF can be fitted by a single Schechter func- 
tion, with M* K = -24.96 ± 0.10 and a = -1.42 ± 0.03 in 
agreement with previous works of comparable depth. 

- The isT-band LF faint-end slope becomes steeper from 
high- to low-density environments varying from -1.33 to - 
1.49, being inconsistent at the 2a c.l. (see Tab[5]l, indicating 
that the faint galaxy population increases in low-density en- 
vironments. Such an environmental dependence confirms our 
finding for the optical LFs derived in the same supercluster 
regions although the changes in slope are less dramatic at 
NIR wavebands. 

- The observed trend of the galaxy SMF presents a slope in 
agreement with pre vious w orks concern ing th e SMF of field 
galaxies (Cole et al. l200ll ; Panter et al. 120041 '). The value of 
M* is hig her wi th respect to that observed in the field (e.g. 
Bell et al. l2003h . as expected for cluster environment. The 
SMF of supercluster galaxies is characterised by an excess of 
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massive galaxies that is associated to the cluster BCGs. Dis- 
crepancies with previous work that observed a strong faint- 
end upturn (Baldry et al. 120081 and Jenkins et al. 120071 ) can 
be related to the different mass ranges investigated and/or 
environmental differences in the analysed structures. 

- Differently from the LF no environment effect is found in 
the slope of the SMFs. On the other hand, the Ai* increase 
from low- to high-density regions and the excess of galaxies 
at the bright-end is also dependent on the environment. This 
tre nd is i n general agreement with the results of Baldry et 
al. (|2006l ). 

In order to interpret our findings, we use the Millennium 
simulation which produce DM halo and galaxy catalogues 
based on SAMs. The cluster NIR LFs obtained using the 
simulated catalogues do not show any significant variation 
with cluster-centric radius, thus suggesting that the varia- 
tions observed in the LFs of the Shapley supercluster are not 
driven by variations in the merging histories of the galaxies, 
but are likely related to processes such as tidal stripping or 
harassment of infalling galaxies. 

By comparing the effect of environment at optical and 
NIR wavebands in shaping the LFs and taking into account 
that the slope of the galaxy SMF is invariant with respect 
to the environment, it seems that the physical mechanism 
responsible for the transformation of galaxies properties in 
different environment are related to the quenching of the 
star formation rather than mass-loss. 

This suggests that the mechanism responsible for shap- 
ing the LF and SMF is partially related to the mass loss due 
to tidal stripping or galaxy harassment, but gas stripping by 
the ICM can also affect the galaxy population removing the 
cold gas supply and thus rapidly terminating ongoing star- 
formation. These mechanisms all require a dense ICM and so 
their evolutionary effects on massive galaxies are limited to 
the cores of clusters, but can extend to poorer environments 
for dwarf galaxies which are easier to strip. The infalling 
galaxies are probably late-type (see MMH06) that are af- 
fected by gas stripping entering in the supercluster. On the 
other hand, the depth of the NIR survey could affect the 
present results by not allowing us to detect an upturn of 
the SMF which can be detect ed at a mass range lower than 
reached here (see Jenkins et al. l2007I ). In order to identify the 
mechanisms which drive galaxy evolution in the supercluster 
environment we are undertaking a survey with the integral 
field spectrograph WiFeS which will provide a unique data 
set to investigate in details the stellar populations and kine- 
matics for a subsample of the Shapley galaxies. 
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